Medical image diagnostic apparatus, ultrasonic diagnostic apparatus, medical imaging system, and imaging control method

ABSTRACT

The medical image diagnostic apparatus according to the embodiment includes processing circuitry. The processing circuitry is configured to acquire a medical image acquired by performing an imaging of a target in a subject and position data corresponding to the medical image. The processing circuitry is configured to determine, based on the acquired medical image and the acquired position data, an unimaged region of the target that is not included in the medical image.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2019-179248, filed on Sep. 30, 2019, andJapanese Patent Application No. 2020-157597, filed on Sep. 18, 2020, theentire contents of each of which are incorporated herein by reference.

FIELD

An embodiment disclosed in the specification and the like relates to amedical image diagnostic apparatus, an ultrasonic diagnostic apparatus,a medical imaging system, and an imaging control method.

BACKGROUND

There is a medical image diagnostic apparatus that generates medicalimage data in which a body tissue of a subject is imaged. Examples ofthe medical image diagnostic apparatus include an ultrasonic diagnosticapparatus, an X-ray computed tomography (CT) apparatus, and a magneticresonance imaging (MRI) apparatus. The ultrasonic diagnostic apparatustransmits ultrasonic waves from an ultrasonic probe into the subject,generates echo signals based on reflected waves, and acquires a desiredultrasonic image by image processing. The X-ray CT apparatus generates aCT image such as an axial tomographic image of a subject based onelectric signals based on X-rays detected by an X-ray detector byirradiating the subject with X-rays. The MRI apparatus is an apparatusthat provides a static magnetic field for a subject and generatesinternal information of the subject as an MRI image based on highfrequency pulses applied to the subject.

In the image acquisition by the medical image diagnostic apparatus,there is a case where only a part of the entire region of a target (forexample, an organ) to be imaged in the subject is visualized. As aresult, some data in the entire region to be imaged may be missing fromthe image. There is no problem if the data-missing area does not appearin the target region, but the data-missing area may appear in the targetregion. When the data-missing area appears in the target region, thedata-missing area in the target region is an unimaged target region. Insuch a case, there is also a method of acquiring an unimaged targetregion by making up for the data-missing area with another image.However, there may be a case where there is no other image having nodata-missing area, or the data-missing area is too large to make up forit.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram showing a configuration of an ultrasonicdiagnostic apparatus as an example of a medical image diagnosticapparatus according to an embodiment.

FIG. 2 is a diagram for explaining position data of an ultrasonic probein the ultrasonic diagnostic apparatus as the example of the medicalimage diagnostic apparatus according to the embodiment.

FIG. 3 is a block diagram showing functions of the ultrasonic diagnosticapparatus according to the embodiment.

FIGS. 4A to 4G are diagrams for explaining the cause of a data-missingarea when performing continuous imaging with the ultrasonic probe in theultrasonic diagnostic apparatus as the example of the medical imagediagnostic apparatus according to the embodiment.

FIGS. 5A and 5B are diagrams for explaining the cause of a data-missingarea when performing intermittent imaging with the ultrasonic probe inthe ultrasonic diagnostic apparatus as the example of the medical imagediagnostic apparatus according to the embodiment.

FIGS. 6A and 6B are diagrams showing the concept of the target in thesubject and data areas included in ultrasonic image data of multiplecross-sections in the ultrasonic diagnostic apparatus as the example ofthe medical image diagnostic apparatus according to the embodiment.

FIG. 7 is an explanatory diagram showing an example of a data flowduring learning in the ultrasonic diagnostic apparatus as the example ofthe medical image diagnostic apparatus according to the embodiment.

FIG. 8 is an explanatory diagram showing an example of a data flowduring operation in the ultrasonic diagnostic apparatus as the exampleof the medical image diagnostic apparatus according to the embodiment.

FIG. 9 is a diagram showing a first operation example of the ultrasonicdiagnostic apparatus as a flowchart, the apparatus being the example ofthe medical image diagnostic apparatus according to the embodiment.

FIG. 10 is a diagram for explaining a method of determining the bodysurface coordinate and posture of the ultrasonic probe for imaging theunimaged organ region due to data-missing in the ultrasonic diagnosticapparatus as the example of the medical image diagnostic apparatusaccording to the embodiment.

Each of FIGS. 11A to 11C is a diagram showing a display example ofinformation regarding the unimaged target region due to data-missing inthe ultrasonic diagnostic apparatus as the example of the medical imagediagnostic apparatus according to the embodiment.

FIG. 12 is a diagram showing, as a flowchart, a second operation exampleof the ultrasonic diagnostic apparatus as the example of the medicalimage diagnostic apparatus according to the embodiment.

FIG. 13 is a schematic diagram showing a configuration of a medicalimage system including the ultrasonic diagnostic apparatus as an exampleof a medical image diagnostic apparatus according to the secondmodification.

DETAILED DESCRIPTION

A medical image diagnostic apparatus, an ultrasonic diagnosticapparatus, a medical imaging system, and an imaging control methodaccording to an embodiment will be described with reference to theaccompanying drawings.

The medical image diagnostic apparatus according to the embodimentincludes processing circuitry. The processing circuitry is configured toacquire a medical image acquired by performing an imaging of a target ina subject and position data corresponding to the medical image. Theprocessing circuitry is configured to determine, based on the acquiredmedical image and the acquired position data, an unimaged region of thetarget that is not included in the medical image.

Examples of the medical image diagnostic apparatus according to theembodiment include an ultrasonic diagnostic apparatus, an X-ray CTapparatus, and an MRI apparatus. The X-ray CT apparatus includes ascanner and a medical image processing apparatus. The X-ray CT apparatusdetects X-rays by an X-ray detector by irradiating the subject withX-rays in the scanner, and generates an electric signal, therebygenerates a CT image such as an axial tomographic image of the subjectbased on the received electric signals in the medical image processingapparatus. The MRI apparatus includes a scanner and a medical imageprocessing apparatus. The MRI apparatus provides a static magnetic fieldformed by the scanner for a subject, applies a high-frequency pulse tothe subject in the scanner, and receives MR signals, thereby generatesan MRI image of the subject based on the MR signal in the medical imageprocessing apparatus.

Hereinafter will be described as an example of an ultrasonic diagnosticapparatus as the medical image diagnostic apparatus, but not limited tothat case.

1. Ultrasonic Diagnostic Apparatus

FIG. 1 is a schematic diagram showing a configuration of an ultrasonicdiagnostic apparatus as an example of a medical image diagnosticapparatus according to an embodiment.

FIG. 1 shows an ultrasonic diagnostic apparatus 1 as an example of amedical image diagnostic apparatus according to an embodiment. Theultrasonic diagnostic apparatus 1 includes a main body 10 as a medicalimage processing apparatus, an ultrasonic probe 30 as a scanner, and aposition sensor 40. In addition, not only the main body 10 may bereferred to as an “ultrasonic diagnostic apparatus”, the main body 10and at least one of the ultrasonic probe 30 and the position sensor 40may also be referred to as an “ultrasonic diagnostic apparatus”. In thefollowing description, the case where the ultrasonic diagnosticapparatus 1 includes both the ultrasonic probe 30 and the positionsensor 40 in addition to the main body 10 will be described.

The main body 10 of the ultrasonic diagnostic apparatus 1 includes atransmission/reception (T/R) circuit 11, a B-mode processing circuit 12,a Doppler processing circuit 13, an image generating circuit 14, animage memory 15, a network interface 16, processing circuitry 17, a mainmemory 18, an input interface 19, and a display 20. The input interface19 and the display 20 may be provided outside the main body 10 or may beprovided outside the ultrasonic diagnostic apparatus 1. The circuits 11to 14 are configured by application specific integrated circuits (ASIC)or the like. However, the invention is not limited to this case, and allor part of the functions of the circuits 11 to 14 may be realized by theprocessing circuitry 17 executing a computer program.

The T/R circuit 11 has a transmitting circuit and a receiving circuit(not shown). Under the control of the processing circuitry 17, the T/Rcircuit 11 controls transmission directivity and reception directivityin transmission and reception of ultrasonic waves. The case where theT/R circuit 11 is provided in the main body 10 will be described, butthe T/R circuit 11 may be provided in the ultrasonic probe 30, or may beprovided in both of the main body 10 and the ultrasonic probe 30. TheT/R circuit 11 is one example of a transmitter-and-receiver.

The transmitting circuit has a pulse generating circuit, a transmissiondelay circuit, a pulsar circuit, etc., and supplies drive signals toultrasonic transducers. The pulse generating circuit repeatedlygenerates rate pulses for forming transmission ultrasonic waves at apredetermined rate frequency. The transmission delay circuit gives adelay time to each rate pulse generated by the pulse generating circuit.The delay time for each transmission ultrasonic is necessary forfocusing ultrasonic waves generated from the ultrasonic transducers ofthe ultrasonic probe 30 into one beam and determining the transmissiondirectivity. In addition, the pulsar circuit applies the drive pulses tothe ultrasonic transducers at timings based on the rate pulses. Thetransmission delay circuit arbitrarily adjusts the transmissiondirection of the ultrasonic beam transmitted from a piezoelectrictransducer surface by changing the delay time given to each rate pulse.

The receiving circuit includes an amplifier circuit, an analog todigital (A/D) converter, an adder, and the like. The receiving circuitreceives echo signals received by the ultrasonic transducers andperforms various processes on the echo signals to generate echo data.The amplifier circuit amplifies the echo signals for each channel andperforms the gain correction processing. The A/D converter performs A/Dconversion of the gain-corrected echo signals, and gives a delay timenecessary for determining reception directivity to digital data. Theadder performs the addition processing on the echo signals processed bythe A/D converter to generate echo data. With the addition processing ofthe adder, the reflection component from the direction corresponding toeach reception directivity of the echo signals is emphasized.

Under the control of the processing circuitry 17, the B-mode processingcircuit 12 receives the echo data from the receiving circuit, performslogarithmic amplification, envelope detection processing and the like,thereby generate data (two-dimensional (2D) or three-dimensional (3D)data) which signal intensity is presented by brightness of luminance.This data is generally called “B-mode data”. The B-mode processingcircuit 12 is one example of a B-mode processer.

The B-mode processing circuit 12 may change the frequency band to bevisualized by changing the detection frequency using filteringprocessing. By using the filtering processing function of the B-modeprocessing circuit 12, harmonic imaging such as the contrast harmonicimaging (CHI) or the tissue harmonic imaging (THI) is performed. Thatis, the B-mode processing circuit 12 may separate the reflected wavedata into harmonic data (or sub-frequency data) and fundamental wavedata in a subject into which the contrast agent is injected. Theharmonic data (or sub-frequency data) corresponds to reflected wave datawith a harmonic component whose reflection source is the contrast agent(microbubbles or bubbles) in the subject. The fundamental wave datacorresponds to reflected wave data with a fundamental wave componentwhose reflection source is tissue in the subject. The B-mode processingcircuit 12 generates B-mode data for generating contrast image databased on the reflected wave data (reception signals) with the harmoniccomponent, and generates B-mode data for generating fundamental waveimage data based on the reflected wave data (reception signals) with thefundamental wave component.

Further, in the THI by using the filter processing function of theB-mode processing circuit 12, it is possible to separate the harmonicdata or the sub-frequency data from the reflected wave data of thesubject. The harmonic data or the sub-frequency data is reflected wavedata (received signals) with a harmonic component. Then, the B-modeprocessing circuit 12 can generate B-mode data for generating the tissueimage data based on the reflected wave data (received signals) with theharmonic component from which the noise component is removed.

Further, when performing the CHI or THI harmonic imaging, the B-modeprocessing circuit 12 can extract the harmonic component by a methoddifferent from the method using the above-described filter processing.In harmonic imaging, the amplitude modulation (AM: Amplitude Modulation)method, the phase modulation (PM: Phase Modulation) method, or animaging method called the AM-PM method, which is a combination of the AMmethod and the PM method, is performed. In the AM method, the PM method,and the AMPM method, ultrasonic waves having different amplitudes andphases are transmitted a plurality of times with respect to the samescanning line. As a result, the T/R circuit 11 generates and outputsmultiple reflected wave data (received signals) on each scanning line.Then, the B-mode processing circuit 12 extracts harmonic components bysubjecting the multiple reflected wave data (received signals) of eachscanning line to addition/subtraction processing according to themodulation method. Then, the B-mode processing circuit 12 performsenvelope detection processing or the like on the reflected wave data(received signals) with the harmonic component to generate the B-modedata.

For example, when the PM method is performed, the T/R circuit 11 causesthe scanning lines set by the processing circuitry 17 to transmit oneset of ultrasonic wave having an inverted phase polarity, such as (−1,1), with the same amplitude on each scanning line. Then, the T/R circuit11 generates a reception signal by transmission of “−1” and a receptionsignal by transmission of “1”, and the B-mode processing circuit 12 addsthese two reception signals. As a result, the fundamental wave componentis removed, and a signal in which the second harmonic component ismainly left is generated. Then, the B-mode processing circuit 12performs envelope detection processing or the like on this signal togenerate B-mode data regarding the THI or CHI.

Alternatively, for example, in the THI, a method of performingvisualization using a second harmonic component and a difference tonecomponent included in a received signal is put into practical use. Inthe visualization method using the difference sound component, forexample, a transmission ultrasonic wave having a composite waveform istransmitted from the ultrasonic probe 30. The composite waveform is acomposition of a first fundamental wave having a center frequency of“f1” and a second fundamental wave having a center frequency of “f2”with a center frequency higher than “f1”. This combined waveform is awaveform acquired by combining the waveform of the first fundamentalwave and the waveform of the second fundamental wave which phases areadjusted with each other such that a difference tone component havingthe same polarity as the second harmonic component is generated. The T/Rcircuit 11 transmits the transmission ultrasonic wave of the compositewaveform, e.g. twice, while inverting the phase. In such a case, forexample, the B-mode processing circuit 12 adds the two received signalsto remove the fundamental wave component, extracts the harmoniccomponent in which the difference tone component, and the secondharmonic component are mainly left, and performs envelope detectionprocessing and the like.

Under the control of the processing circuitry 17, the Doppler processingcircuit 13 frequency-analyzes the phase information from the echo datafrom the receiving circuit, thereby generating data (2D or 3D data)acquired by extracting multiple dynamic data of moving target such asaverage speed, dispersion, power and the like. This data is generallycalled “Doppler data”. In the embodiment, the moving target refers to,for example, blood flow, tissue such as heart wall, or contrast agent.The Doppler processing circuit 13 is one example of a Doppler processer.

Under the control of the processing circuitry 17, the image generatingcircuit 14 generates an ultrasonic image as image data presented in apredetermined luminance range based on the reception signals received bythe ultrasonic probe 30. For example, the image generating circuit 14generates, as the ultrasonic image, a B-mode image in which theintensity of the reflected wave is presented by luminance based on 2DB-mode data generated by the B-mode processing circuit 12. In addition,the image generating circuit 14 generates, as the ultrasonic image, acolor Doppler image based on 2D Doppler data generated by the Dopplerprocessing circuit 13. The color Doppler image includes an average speedimage presenting moving state information, a dispersion image, a powerimage, or a combination image thereof. The image generating circuit 14is one example of an image generator.

In the embodiment, the image generating circuit 14 generally converts(scan-converts) a scanning line signal sequence of ultrasonic scanninginto a scanning line signal sequence of a video format for a televisionor the like, and generates ultrasonic image data for display.Specifically, the image generating circuit 14 generates the ultrasonicimage data for display by performing coordinate conversion according tothe ultrasonic scanning mode of the ultrasonic probe 30. The imagegenerating circuit 14 performs various image processes other than thescan conversion. For example, the image generating circuit 14 performsimage processing (smoothing processing) for regenerating an averageluminance image using multiple image frames after scan conversion, orimage processing using a differential filter in the image (processingfor enhancing edges) and the like. Further, the image generating circuit14 combines character information of various parameters, scales, bodymarks, and the like with the ultrasonic image data.

That is, the B-mode data and the Doppler data are the ultrasonic imagedata before the scan conversion processing. The data generated by theimage generating circuit 14 is the ultrasonic image data for displayafter the scan conversion processing. The B-mode data and the Dopplerdata are also called raw data. The image generating circuit 14 generates2D ultrasonic image data for display based on the 2D ultrasonic imagedata before the scan conversion processing.

The image memory 15 includes multiple memory cells in one frame in twoaxial directions, and includes a 2D memory which is a memory providedwith multiple frames. The image memory 15, as the 2D memory, stores oneframe or multiple frames of the ultrasonic image as 2D image datagenerated by the image generating circuit 14 under the control of theprocessing circuitry 17. The image memory 15 is one example of astorage.

Further, the image generating circuit 14 performs coordinate conversionon the 3D B-mode data generated by the B-mode processing circuit 12,thereby generates 3D B-mode image data. The image generating circuit 14also performs coordinate conversion on the 3D Doppler data generated bythe Doppler processing circuit 13, thereby generates 3D Doppler imagedata. The image generating circuit 14 generates “3D B-mode image data or3D Doppler image data” as “3D ultrasonic image data (volume data)”.

The image memory 15 may include a 3D memory which is a memory havingmultiple memory cells in three axis directions (X-axis, Y-axis, andZ-axis directions). The image memory 15, as the 3D memory, stores themultiple ultrasonic images generated by the image generating circuit 14as the volume data under the control of the processing circuitry 17.

Then, in order to generate various 2D image data so as to display thevolume data stored in the 3D memory on the display 20, the imagegenerating circuit 14 performs processing for displaying the volume dataon the 2D display and processing for displaying the 3D datathree-dimensionally, with respect to the volume data. The imagegenerating circuit 14 performs the processing such as volume rendering(VR) processing, surface rendering (SR) processing, MIP (MaximumIntensity Projection) processing, MPR (Multi Planer Reconstruction)processing, etc.

The network interface 16 implements various information communicationprotocols according to the network form. The network interface 16connects the ultrasonic diagnostic apparatus 1 and other devices such asan external medical image managing apparatus and a medical imageprocessing apparatus according to these various protocols. An electricalconnection or the like via an electronic network is applied to thisconnection. In the embodiment, the electronic network generally refersto an information communication network using telecommunicationstechnology. The electronic network includes a wired/wireless hospitalbackbone local area network (LAN) and the internet network, as well as atelephone communication line network, an optical fiber communicationnetwork, a cable communication network, a satellite communicationnetwork, or the like.

Further, the network interface 16 may implement various protocols fornon-contact wireless communication. In this case, the main body 10 candirectly transmit/receive data to/from the ultrasonic probe 30, forexample, without going through the network. The network interface 16 isone example of a network connector.

The processing circuitry 17 may refer to a dedicated or general-purposecentral processing unit (CPU), microprocessor unit (MPU), graphicsprocessing unit (GPU), or the like. The processing circuitry 17 mayrefers to an ASIC, a programmable logic device, or the like. Theprogrammable logic device is, for example, a simple programmable logicdevice (SPLD), a complex programmable logic device (CPLD), and a fieldprogrammable gate array (FPGA).

Further, the processing circuitry 17 may be constituted by a singlecircuit or a combination of independent circuit elements. In the lattercase, the main memory 18 may be provided individually for each circuitelement, or a single main memory 18 may store programs corresponding tothe functions of the circuit elements. The processing circuitry 17 isone example of a processor.

The main memory 18 is constituted by a semiconductor memory element suchas a random-access memory (RAM), a flash memory, a hard disk, an opticaldisk, or the like. The main memory 18 may be constituted by a portablemedium such as a universal serial bus (USB) memory and a digital videodisk (DVD). The main memory 18 stores various processing programs(including an operating system (OS) and the like besides the applicationprogram) used in the processing circuitry 17 and data necessary forexecuting the programs. In addition, the OS may include a graphical userinterface (GUI) which allows the operator to frequently use graphics todisplay information on the display 20 to the operator and can performbasic operations by the input interface 19. The main memory 18 is oneexample of a storage.

The input interface 19 includes an input device operable by an operator,and a circuit for inputting a signal from the input device. The inputdevice may be a trackball, a switch, a mouse, a keyboard, a touch padfor performing an input operation by touching an operation surface, atouch screen in which a display screen and a touch pad are integrated, anon-contact input circuit using an optical sensor, an audio inputcircuit, and the like. When the input device is operated by theoperator, the input interface 19 generates an input signal correspondingto the operation and outputs it to the processing circuitry 17. Theinput interface 19 is one example of an input unit.

The display 20 is constituted by a general display output device such asa liquid crystal display or an organic light emitting diode (OLED)display. The display 20 displays various kinds of information under thecontrol of the processing circuitry 17. The display 20 is one example ofa display unit.

The ultrasonic probe 30 of the ultrasonic diagnostic apparatus 1 is ascanner including minute transducers (piezoelectric elements) on thefront surface. The ultrasonic probe 30 transmits/receives ultrasonicwaves to/from a target in a subject (for example, a patient). Eachtransducer is an electroacoustic conversion element, and convertselectric pulses into ultrasonic pulses during transmission. Further, theultrasonic probe 30 has a function of converting reflected waves intoelectric signals (received signals) at the time of reception. Theultrasonic probe 30 is small and lightweight, and is connected to themain body 10 via a cable (or wireless communication).

The ultrasonic probe 30 is classified into types such as a linear type,a convex type, a sector type, etc. depending on differences in scanningsystem. Further, the ultrasonic probe 30 is classified into a 1D arrayprobe in which transducers are arrayed in a one-dimensional (1D) mannerin the azimuth direction, and a 2D array probe in which transducers arearrayed in a 2D manner in the azimuth direction and in the elevationdirection, depending on the array arrangement dimension. The 1D arrayprobe includes a probe in which a small number of transducers arearranged in the elevation direction.

In the embodiment, when performing 3D scanning, that is, volumescanning, a 2D array probe having a scanning method such as a lineartype, a convex type, or a sector type is used as the ultrasonic probe30. Alternatively, when the volume scan is performed, a 1D probe havinga scan method such as a linear type, a convex type, or a sector type andhaving a mechanism that mechanically swings in the elevation directionis used as the ultrasonic probe 30. The latter probe is also called amechanical 4D probe.

The position sensor 40 detects multiple position data of the ultrasonicprobe 30 in time series and outputs the multiple position data to themain body 10. The position sensor 40 includes a sensor attached to theultrasonic probe 30 and a sensor provided separately from the ultrasonicprobe 30. The latter sensor is an optical sensor that imagescharacteristic points of the ultrasonic probe 30 as a measurement targetfrom multiple positions, and detects each position of the ultrasonicprobe 30 on the principle of triangulation.

The type of the position sensor 40 attached to the ultrasonic probe 30detects position data of itself, and outputs the position data to themain body 10. The position data of the position sensor 40 can also beregarded as the position data of the ultrasonic probe 30. The positiondata of the ultrasonic probe 30 includes a coordinate (X, Y, Z) of theultrasonic probe 30 and a tilt angle (posture) from each axis. Forexample, the posture of the ultrasonic probe 30 can be detected by amagnetic field transmitter (not shown) sequentially transmittingtriaxial magnetic fields while the position sensor 40 sequentiallyreceiving the magnetic fields.

Further, the position sensor 40 may be a so-called nine-axis sensor. Thenine-axis sensor includes at least one of: a three-axis gyro sensor thatdetects three-axis angular velocity in a 3D space; a three-axisacceleration sensor that detects three-axis acceleration in a 3D space;and a three-axis geomagnetic sensor that detects three-axis geomagnetismin a 3D space. The position sensor 40 is not an essential component.

FIG. 2 is a diagram for explaining position data of the ultrasonic probe30.

FIG. 2 shows three orthogonal directions with respect to the ultrasonicprobe 30, that is, the U-axis direction, the V-axis direction, and theW-axis direction. The U-axis direction is defined as the transducerarray direction, that is, an azimuth direction. The V-axis direction isdefined by the depth direction, that is, the direction orthogonal to theU-axis direction and the W-axis direction. The W-axis direction isdefined as an elevation direction. Regarding the ultrasonic probe 30which coordinate and posture are defined by the U-axis direction, theV-axis direction, and the W-axis direction. The ultrasonic probe 30 isarranged and operated to move arbitrarily in the XYZ space in which thesubject is provided.

Subsequently, functions of the ultrasonic diagnostic apparatus 1 will bedescribed.

FIG. 3 is a block diagram showing functions of the ultrasonic diagnosticapparatus 1.

The processing circuitry 17 reads and executes a computer program storedin the main memory 18 or directly incorporated in the processingcircuitry 17, thereby realizes an acquiring function 171, a derivingfunction 172, a determining function 173, an output control function174, and a moving control function 175. Hereinafter, a case where thefunctions 171 to 175 function as software will be described as anexample. However, all or a part of the functions 171 to 175 may beprovided in the ultrasonic diagnostic apparatus 1 as a circuit such asthe ASIC. All or part of the deriving function 172 and the determiningfunction 173 may be realized by an external apparatus connected via thenetwork N.

The acquiring function 171 has a function of controlling the T/R circuit11, the B-mode processing circuit 12, the Doppler processing circuit 13,the image generating circuit 14, and the like to execute the imaging ofa target in the subject using the ultrasonic probe 30, thereby acquiringultrasonic image data as the medical image data. Specifically, theacquiring function 171 acquires M-mode image data, B-mode image data,Doppler image data, and the like as the ultrasonic image data. Further,the acquiring function 171 includes a function of acquiring positiondata corresponding to the ultrasonic image data (e.g., position data ofthe ultrasonic probe 30 and position data of a cross-section based onthe position of the ultrasonic probe 30).

For example, the acquiring function 171 acquires ultrasonic image dataof multiple cross-sections as the medical image data of multiplecross-sections. Further, the acquiring function 171 acquires theposition data corresponding to each of the ultrasonic image data ofcross-sections. Hereinafter, a case where the acquiring function 171acquires ultrasonic image data of the cross-sections will be described.The acquiring function 171 is one example of an acquiring unit.

In the embodiment, the ultrasonic image data of each cross-sectionacquired by the acquiring function 171 includes a data area that isvisualized and a data-missing area that is not visualized and isdifficult to make up for it. The data area means a range of anultrasonic beam (raster) having a length that the transmitted wave canreach.

FIGS. 4A to 4G are diagrams for explaining the cause of the data-missingarea when performing continuous imaging with the ultrasonic probe 30.

FIG. 4A shows ideal data areas P when the ultrasonic probe 30 isoperated in the positive direction of the W-axis. In FIG. 4A, the dataarea P is almost evenly spread over the entire target region R, and adata-missing area where the data area P does not exist does not appear.For example, the target is an organ and the target region is an organregion corresponding to the organ. In the embodiment, the organ is aunit that constitutes the body of an animal multicellular organism, andrefers to an aggregation of tissues that cooperate to perform a certainfunction. The organ includes, for example, brain, heart, lung, liver,pancreas, stomach, intestine, blood vessel, nerve, and the like.

FIG. 4B shows each data area P in the case where the operation speed ofthe ultrasonic probe 30 in the positive direction of the W-axis is notconstant with respect to FIG. 4A. In FIG. 4B, a data-missing areaappears in which the data area P does not exist in the target region Rwhere the space between the adjacent data areas P are relatively large.

FIG. 4C shows each data area P in the case where a tissue having a largedifference in acoustic impedance is included with respect to FIG. 4A. InFIG. 4C, in the target region R, a data-missing area appears in whichthe data area P does not exist in the central position on the negativeside of the V-axis. In a tissue with a large difference in acousticimpedance, because the ultrasonic beam in the data area P is greatlyattenuated, the length of the data area P passing through the tissue isshortened.

FIG. 4D shows each data area P in the case where imaging is stoppedduring the process of imaging the target region R corresponding to thetarget to be imaged, with respect to FIG. 4A. In FIG. 4D, in the targetregion R, a data-missing area appears where the data area P does notexist in a position on the positive direction side of the W-axis.

FIG. 4E shows each data area P in the case where the posture of theultrasonic probe 30 during operation is not constant (the VW-axisrotates in the VW-plane) with respect to FIG. 4A. In FIG. 4E, in thetarget region R, a data-missing area appears in which the data area Pdoes not exist in positions in the target region R where the spacebetween the adjacent data areas P are relatively large.

FIG. 4F shows each data area P in the case where the operation of theultrasonic probe 30 is not linear (W-axis changes in the U-axisdirection) with respect to FIG. 4A. In FIG. 4F, in the target region R,a data-missing area appears in which the data area P does not exist in aposition on the positive direction side of the U-axis and on thenegative direction side of the W-axis.

FIG. 4G shows each data area P in the case where the rotation angle ofthe ultrasonic probe 30 (UW-axis rotates in UW plane) during theoperation is not constant with respect to FIG. 4A. In FIG. 4G, in thetarget region R, a data-missing area appears in which the data area Pdoes not exist in positions where the space between the adjacent dataareas P are relatively large.

In each of the cases shown in FIGS. 4B to 4G, a data-missing areaappears where the data area P does not exist in the target region R ascompared with the case of FIG. 4A.

FIGS. 5A and 5B are diagrams for explaining the cause of thedata-missing area when performing intermittent imaging with theultrasonic probe 30.

FIG. 5A shows each data area P in the case where a tissue having a largedifference in acoustic impedance is not included. FIG. 5B shows eachdata area P in the case where a tissue having a large difference inacoustic impedance is included with respect to FIG. 5A. In a tissuewhere the difference in acoustic impedance is large, because theultrasonic beam in the data area P is greatly attenuated, the length ofthe data area P passing through the tissue is shortened.

In each of the left and right cases shown in FIG. 5B, the data-missingarea appears in the target region R where the data area P does not existbecause the data area P is shorter in comparison with the case of FIG.5A. As a result, the unimaged target region Q appears in the targetregion R. Note that a region excluding the unimaged target region Q inthe target region R is an imaged target region. That is, the targetregion R is composed of the unimaged target region Q and the imagedtarget region.

Note that the data-missing area also appears in a complicated manner byany combination of FIGS. 4B to 4G and 5B. As a result, in the targetregion R, the unimaged target region appears in a complicated manner.

Returning to the description of FIG. 3 , the deriving function 172 has afunction of deriving a target shape and an imaged target region in thesubject from multiple ultrasonic image data and multiple position dataacquired by the acquiring function 171. For example, the derivingfunction 172 includes a function of deriving an organ shape and theimaged organ region in the subject from the ultrasonic image data ofmultiple cross-sections and their position data. The deriving function172 is one example of a deriving unit.

Further, the deriving function 172 arranges the ultrasonic image dataacquired by the acquiring function 171 in the image memory 15 as the 3Dmemory on the basis of the position data acquired by the position sensor40.

FIGS. 6A and 6B are diagrams showing the concept of the target in thesubject and data areas P included in ultrasonic image data of multiplecross-sections.

FIG. 6A is a diagram showing a liver region R1 as a target region Racquired by transmitting and receiving ultrasonic waves to and from theliver which is the target in the subject. FIG. 6B shows data areas Pwhen the ultrasonic probe 30 is applied to multiple positions on thebody surface of the subject and the ultrasonic waves are transmitted andreceived to and from the multiple positions with respect to the livershown in FIG. 6A.

As conceptually shown in FIG. 6B, even if ultrasonic waves aretransmitted from any position of the liver R1 at any angle, the entireliver region R1 cannot be acquired as the data areas P that can bevisualized. The gray portion shown in FIG. 6A but missing in FIG. 6B arethe unimaged target region that appears due to data-missing. Thedata-missing occurs due to what has been described with reference toFIGS. 4A to 4G and FIGS. 5A and 5B.

Therefore, the deriving function 172 performs a process of deriving ashape of the organ such as the liver in the subject from the ultrasonicimage data of cross-sections arranged three-dimensionally and fromposition data, thereby derives the organ shape and the imaged organregion. Then, the determining function 173, which will be describedlater, determines a region excluding the derived imaged organ regionfrom the entire derived liver region R1 as an unimaged target region.The unimaged target region is a portion of the entire target region thatcannot be visualized based on the data-missing area where the data areasP are insufficient. The unimaged target region means a region thatshould be continuously imaged in the present examination.

The deriving function 172 may use, for example, a database in whichultrasonic image data and an organ shape are associated with each otherin the deriving process of the organ shape in the subject. The organshape is further associated with not only the information indicating thegeneral shape of the entire organ and the organ name but also:trajectory information indicating how to move the ultrasonic probe 30 toimage the entire organ; various images acquired when the target organwas imaged in the past; and information regarding the position data ofthe various images. The deriving function 172 may use machine learningfor the process of deriving the shape of the organ in the subject.Further, as machine learning, deep learning using a multilayer neuralnetwork such as CNN (Convolutional Deep Belief Network) andconvolutional deep belief network (CDBN) may be used.

Hereinafter, an example of a case where the deriving function 172includes the neural network Na, and where the entire organ shapeincluded in the ultrasonic image data is derived from the partial organshape included in the ultrasonic image data using deep learning will bedescribed.

FIG. 7 is an explanatory diagram showing an example of a data flowduring learning.

The deriving function 172 inputs a large number of training data andperforms learning to sequentially update the parameter data Pa. Thetraining data is made up of a set of multiple ultrasonic image data(e.g., arbitrary cross-section data forming volume data) S1, S2, S3, . .. as training input data and organ shapes T1, T2, T3, . . .corresponding to each arbitrary cross-section data. The multipleultrasonic image data S1, S2, S3, . . . constitutes a training inputdata group Ba. The organ shapes T1, T2, T3, . . . constitutes a trainingoutput data group Ca.

The deriving function 172 updates the parameter data Pa such that theresult of processing the multiple ultrasonic image data S1, S2, S3, . .. by the neural network Na approaches the organ shapes T1, T2, T3, . . .every time the training data is input, that is so-called learning.Generally, when the rate of change of the parameter data Pa convergeswithin the threshold value, the learning is determined to be completed.Hereinafter, the parameter data Pa after learning is particularlyreferred to as learned parameter data Pa′.

Note that the type of training input data and the type of input dataduring operation shown in FIG. 8 should match. For example, if the inputdata during operation is ultrasonic image data, the training input datagroup Ba during learning should be ultrasonic image data as well.

The “ultrasonic image data” includes raw data generated by theultrasonic diagnostic apparatus 1. That is, the input data of the neuralnetwork Na may be raw data before scan conversion.

FIG. 8 is an explanatory diagram showing an example of a data flowduring operation.

In operation, the deriving function 172 inputs the ultrasonic image dataSa of the target to be diagnosed, and outputs the organ shape Taincluded in the ultrasonic image data using the learned parameter dataPa′. When outputting the organ shape Ta, not only the shape and organname of the entire organ but also any of: the current coordinate andangle of the ultrasonic probe; the motion of the ultrasonic probe 30 forimaging the unimaged target region, until the entire organ is imaged;and position data of each image in the entire organ may be output.

The neural network Na and the learned parameter data Pa′ form a learnedmodel 19 a. The neural network Na is stored in the main memory 18 in theform of a program. The learned parameter data Pa′ may be stored in themain memory 18 or may be stored in a storage medium connected to theultrasonic diagnostic apparatus 1 via the network N. In this case, thederiving function 172 realized by the processor of the processingcircuitry 17 reads the learned model 19 a from the main memory 18 andexecutes it, thereby generates information on the organ shape includedin the ultrasonic image data. The learned model 19 a may be constructedby an integrated circuit such as an ASIC (Application SpecificIntegrated Circuit) and an FPGA (Field Programmable Gate Array).

Note that supplemental information may be used as input data in additionto the ultrasonic image data so as to improve the accuracy ofdetermination by the deriving function 172. The supplementaryinformation includes at least one of image data regarding the height andweight of the subject to be imaged, other modalities already imaged, andrepresentative model data of gadget.

In this case, at the time of learning, the supplementary information ofeach subject of the multiple ultrasonic image data S1, S2, S3, . . . asthe training input data is also input to the neural network Na as thetraining input data. At the time of operation, the deriving function 172inputs the ultrasonic image data Ba of the target to be diagnosed andthe supplemental information of the subject having the target to thelearned model 19 a read from the main memory 18, thereby outputs theorgan shape Ta included in the ultrasonic image data. By using theultrasonic image data and the supplementary information of the subjectas the input data, it is possible to generate the learned parameter dataPa′ that has been learned according to the type of the subject.Therefore, it is possible to improve the deriving accuracy as comparedwith the case where only the ultrasonic image data is used as the inputdata.

Returning to the description of FIG. 3 , the determining function 173has a function of determining an unimaged target region based on theultrasonic image data of cross-sections and the position data acquiredby the acquiring function 171. The unimaged target region is a region ofthe target that is not included in the ultrasonic image data of thecross-sections. Alternatively, the determining function 173 has afunction of determining an unimaged target region (e.g., unimaged organregion) on the basis of the target shape (e.g., organ shape) derived bythe deriving function 172 and the imaged target region (e.g., organregion). That is, the determining function 173 three-dimensionallycompares the imaged organ shape included in the ultrasonic image datawith the derived organ shape, thereby determines the unimaged organregion resulting from data-missing in the organ region. The determiningfunction 173 is one example of a determining unit.

The output control function 174 has a function of outputting thethree-dimensional information of the unimaged target region determinedby the determining function 173 and/or information for imaging theunimaged target region to the outside of the processing circuitry 17.For example, the output control function 174 includes a display controlfunction 71, a storage control function 72, and a transmission controlfunction 73. The output control function 174 may include at least one ofthe display control function 71, the storage control function 72, andthe transmission control function 73.

The display control function 71 has a function of displaying theultrasonic image data acquired by the acquiring function 171 on thedisplay 20, and a function of displaying information regarding anunimaged target region when the unimaged target region is determined bythe determining function 173 on the display 20. For example, theinformation regarding the unimaged target region is the coordinate(shown as a marker) of the ultrasonic probe 30 on the body surface, theangle, and the pressure at the time of imaging. The coordinate of theultrasonic probe 30 on the body surface is a display of the unimagedorgan region (position/size) with respect to the entire organ, and isnecessary to image the unimaged organ region. An alert can be displayedtogether with the display of an unimaged organ region. The coordinate ofthe ultrasonic probe 30 on the body surface may be directly projected onthe body surface. The display control function 71 is one example of adisplay control unit.

The storage control function 72 has a function of storing thethree-dimensional information of the unimaged target region and theinformation for imaging in the storage medium such as image memory 15when the unimaged target region is determined by the determiningfunction 173. The storage control function 72 is one example of astorage control unit.

The transmission control function 73 has a function of transmitting thethree-dimensional information of the unimaged target region and theinformation for imaging to an outside apparatus (e.g., the medical imagedisplay apparatus 80 shown in FIG. 13 ) of the medical image processingapparatus 10 via the network interface 16 when the unimaged targetregion is determined by the determining function 173. The transmissioncontrol function 73 is one example of a transmission control unit.

The moving control function 175 has a function of controlling anexternal device such as a robot arm when the unimaged target region isdetermined by the determining function 173, thereby moving (includingslide movement, rotation movement, probe posture angle, and probepressure change) the ultrasonic probe 30 which is the scanner. Themoving operation of the ultrasonic probe 30 may be operated by anoperator who holds the ultrasonic probe 30, may be performed by an autoscan performed for the purpose of correcting movement of the subjectsuch as respiratory characteristics, or may be performed by the robotarm scan for the purpose of reducing the operation of the ultrasonicprobe 30 by the operator. The moving control function 175 is one exampleof a moving control unit.

Subsequently, the operation of the ultrasonic diagnostic apparatus 1will be described with reference to FIGS. 9 and 12 .

FIG. 9 is a diagram showing a first operation example of the ultrasonicdiagnostic apparatus 1 as a flowchart. In FIG. 9 , reference numerals inwhich “ST” is attached to numbers indicate steps in the flowchart. Inaddition, in FIG. 9 , a case where the target to be imaged is an organwill be described.

The acquiring function 171 controls the T/R circuit 11, the B-modeprocessing circuit 12, the Doppler processing circuit 13, the imagegenerating circuit 14, and the like and executes the ultrasonic imagingof the target in the subject using the ultrasonic probe 30, therebyacquires ultrasonic image data (step ST1). The display control function71 displays the ultrasonic image data acquired in step ST1 as anultrasonic image on the display 20 (step ST2).

Further, the acquiring function 171 stores the ultrasonic image dataacquired in step ST1 in a three-dimensional arrangement in the 3D memoryof the image memory 15 on the basis of the position data of theultrasonic image data (step ST3). The position sensor 40 is not anindispensable component because the relative change in the coordinateand posture of the ultrasonic probe 30 can be detected by matching thedata area in which the data to be visualized exists. The display controlfunction 71 may display the three-dimensional arrangement of ultrasonicimage data on the display 20. As a result, the operator who takes animage can visually recognize the range of the image taken in the body,particularly in the trunk.

The acquiring function 171 determines whether to finish the ultrasonicimaging (step ST4). The acquiring function 171 may determine to finishthe ultrasonic imaging on the basis of the finish instruction input bythe operator via the input interface 19, or may determine that theultrasonic imaging is finished if the ultrasonic probe 30 is in the airapart from the body surface of the subject after a certain time elapsed.For example, whether the ultrasonic probe 30 is in the air may bedetermined based on the position data of the ultrasonic probe 30.

If it is determined as “NO” in step ST4, that is, if it is determinedthat the ultrasonic imaging for the next cross-section will be performedwithout finishing ultrasonic imaging, the acquiring function 171controls the T/R circuit 11, the B-mode processing circuit 12, theDoppler processing circuit 13, the image generating circuit 14, etc.,executes the ultrasonic imaging using the ultrasonic probe 30, therebyacquires ultrasonic image data for the next cross-section (step ST1). Byrepeating the set of steps ST1 to ST3 based on “NO” in step ST4,ultrasonic image data of cross-sections is arranged in the 3D memory ofthe image memory 15 (shown in FIG. 6B).

On the other hand, if it is determined as “YES” in step ST4, that is, ifit is determined that the ultrasonic imaging is to be finished, thederiving function 172 derives the organ shape of the entire target organin the subject and the imaged organ region on the basis of theultrasonic image data of one or multiple cross-sections arranged in the3D memory of the image memory 15 (step ST5). The determining function173 determines a three-dimensional unimaged organ region on the basis ofthe partial organ shape included in the ultrasonic image data of one ormultiple cross-sections arranged three-dimensionally in step ST3 and theentire organ shape derived in step ST5 (step ST6).

The determining function 173 extracts an organ contour from ultrasonicimage data of cross-sections arranged three-dimensionally, arranges theextracted organ contour in a three-dimensional manner, and collates thearranged one with a 3D model of the entire organ. Then, the determiningfunction 173 determines, as an unimaged organ region, a region acquiredby removing the organ contour included in the already existing data areafrom the organ contour of the 3D model. Note that the 3D model may begenerated by acquiring the organ contour from volume data such as 3D-CTimage data acquired in advance from the same subject (same patient), ormay be a 3D model showing a general organ shape.

The determining function 173 determines whether or not there is anunimaged organ region (step ST7). If it is determined as “YES” in stepST7, that is, if it is determined that there is the unimaged organregion, the display control function 71 displays information regardingthe unimaged organ region on the display 20 for the operator (step ST8).The information regarding the unimaged organ region may be the range ofthe ultrasonic beam determined to fill the unimaged organ region(display example (1) described below), or may include the body surfacecoordinate and posture (display examples (2) to (4) described later) ofthe ultrasonic probe 30 for imaging an unimaged organ region. Further,the display control function 71 displays the information for imaging theunimaged target region, thereby displaying the information regarding theunimaged target region, and/or three-dimensionally displays the unimagedtarget region.

FIG. 10 is a diagram for explaining a method of determining the bodysurface coordinate and posture of the ultrasonic probe 30 for imagingthe unimaged organ region due to data-missing.

FIG. 10 shows the unimaged target region Q shown in FIG. 5B. In the caseof ultrasonic imaging, the display control function 71 may select onehaving a similar imaging area, imaging position, and imaging angle tothe already existing data area P in body surface coordinate and posturethat can image the unimaged target region Q in the shallowest possiblearea (e.g., two probe coordinates at both ends in FIG. 10 ). The displaycontrol function 71 may select the body surface coordinate and postureof the ultrasonic probe 30 displayed for imaging the unimaged targetregion. Note that the body surface coordinate and posture selected maybe one or more. Further, when the position of an anatomy (for example,bone) where the difference in acoustic impedance in the subject is largeis known, the body surface coordinate and posture where the anatomy doesnot have a large difference in acoustic impedance on the cross-sectiondetermined by the body surface coordinate and posture can be selected,or the body surface coordinate and posture can be adjusted by shiftingthe body surface coordinate and posture.

The information regarding the unimaged target region will be shown inthe following display examples (1) to (4).

-   -   (1) A range of the ultrasonic beam determined to fill the        unimaged target region (shown in FIG. 6B) on the 3D model is        displayed.    -   (2) A probe mark of the ultrasonic probe 30 for imaging an        unimaged target region is projected and displayed on the        subject.    -   (3) A probe mark of the ultrasonic probe 30 for imaging the        unimaged target region is displayed on the 3D model of the        subject.    -   (4) An LED lamp or the like is attached to the ultrasonic probe        30 itself, and the moving and rotating directions are displayed        by the lamp.

Each of FIGS. 11A to 11C is a diagram showing a display example ofinformation regarding the unimaged target region due to data-missing.

FIG. 11A shows the above display example (1). FIG. 11B shows the abovedisplay example (2). FIG. 11C shows the above display example (3). InFIG. 11A, the top side of the trapezoid formed by a thick line (theshorter side of the two sides) corresponds to the body surfacecoordinate. In FIG. 11B, the mark M1 is a probe mark projected on thebody surface of the subject, and the mark M2 is a mark of a marker whichis projected on the body surface of the subject and is attached to oneside surface of the ultrasonic probe 30. In FIG. 11C, the mark M1 is aprobe mark on the 3D model displayed on the display 20, and the mark M2is mark of a marker attached to one side surface of the ultrasonic probe30 on the 3D model displayed on the display 20.

Note that the unimaged target region may be projected with the marks M1and M2 on the body surface of the subject. The position of the unimagedtarget region is acquired based on the multiple ultrasonic image dataincluding the imaged target region and their position data. This isbecause it is possible to acquire the position of the entire targetregion based on the multiple ultrasonic image data including the imagedtarget region and their position data.

The operator confirms the information regarding the unimaged targetregion displayed in step ST8, and continues the examination ifnecessary. Specifically, the operator presses the tip of the ultrasonicprobe 30 to the body surface position of the subject corresponding tothe body surface position of the subject displayed on the display 20(top side of the trapezoid). Alternatively, the operator presses the tipof the ultrasonic probe 30 against the mark M1 (shown in FIG. 11B) suchthat the marker matches the mark M2 (shown in FIG. 11B) projected on thebody surface of the subject. Alternatively, the operator presses the tipof the ultrasonic probe 30 against the mark M1 (shown in FIG. 11C) suchthat the marker matches the mark M2 (shown in FIG. 11C) at the bodysurface position of the subject displayed on the display 20. Theoperator presses the ultrasonic probe 30 to match the markers attachedto the ultrasonic probe 30 so as to match the position of the ultrasonicprobe 30 in the rotation direction.

Further, it is possible to display a probe model for imaging theunimaged target region on the 3D model of the subject, thereby displaythe direction in which the ultrasonic probe 30 should be moved androtated with respect to the probe model by an arrow. Note that theunimaged target region may be displayed on the probe model with anarrow.

Returning to the description of FIG. 9 , if it is determined as “NO” instep ST7, that is, if it is determined that there is no unimaged organregion, the examination is finished.

As described above, according to the first operation example of theultrasonic diagnostic apparatus 1, when the data-missing occurs due toinsufficient imaging, it is possible to encourage the operator after theimaging to perform additional imaging of the unimaged target region dueto data-missing. Further, it is also possible to present the operatorwith information regarding the unimaged target region. For example, itis possible to display the unimaged target region itself as theinformation regarding the unimaged region, or to display whichcoordinate and posture the ultrasonic probe 30 should be applied to onthe body surface of the subject. Further, it does not rely oncomplementation using other images to make up for the unimaged targetregion. As a result, it is possible to avoid a situation that additionalimaging should be performed on the unimaged target region is determinedafter the subject leaves the room after the examination.

FIG. 12 is a diagram showing a second operation example of theultrasonic diagnostic apparatus 1 as a flowchart. In FIG. 12 , thereference numerals in which “ST” is attached to numbers indicate thesteps of the flowchart. The second operation example of the ultrasonicdiagnostic apparatus 1 is different from the first operation example ofthe ultrasonic diagnostic apparatus 1 that the unimaged target regionsis determined sequentially. In addition, in FIG. 12 , a case where thetarget to be imaged is an organ will be described.

In the FIG. 12 , the same steps as those shown in FIG. 9 are designatedby the same reference numerals, and the description thereof will beomitted.

The deriving function 172 derives the organ shape of the entire targetorgan in the subject and the imaged organ region from the ultrasonicimage data of one or multiple cross-sections arranged in the 3D memoryof the image memory 15, as in step ST5 shown in FIG. 9 (step ST15). Thedetermining function 173 determines a three-dimensional unimaged organregion on the basis of the ultrasonic image data of one or multiplecross-sections arranged in the 3D memory of the image memory 15, as instep ST6 shown in FIG. 9 (step ST16).

The display control function 71 determines whether or not the ratio ofthe volume or contour of the unimaged organ region to the volume orcontour of the entire organ shape derived in step ST15 is equal to orless than a threshold value (e.g. 20%). If it is determined as “YES” instep ST17, that is, if it is determined that the volume or contour ratioof the unimaged organ region is less than or equal to the thresholdvalue, the display control function 71 displays on the display 20 thatthe data is sufficient with only a little missing data (step ST18).

On the other hand, if it is determined as “NO” in step ST17, that is, ifit is determined that the ratio of the volume of the unimaged organregion exceeds the threshold value, the display control function 71displays on the display 20 information regarding the unimaged organregion for the operator, as in step ST8 shown in FIG. 9 (step ST19).

As described above, according to the second operation example of theultrasonic diagnostic apparatus 1, it is possible to present theoperator the ratio of the volume or contour of the unimaged targetregion to the volume or contour of acquired from the entire derivedtarget shape and with its coordinate and range in real time duringimaging. As a result, it is possible to for the operator to proceed withimaging while confirming the ratio of the unimaged target region.Therefore, it is possible to avoid a situation in which it is determinedthat additional imaging should be performed on the unimaged targetregion after the subject disappears after the examination.

Note that the data-missing due to insufficient imaging is described, butthe present invention is not limited to this case. For example, thepresent invention can be applied to the case where there is an artifacton the ultrasonic image data and a part of the tissue to be visuallyrecognized cannot be visually recognized even if it is not thedata-missing. In this case, the artifact portion on the ultrasonic imagedata can be detected and set the portion as the data-missing area.

After the unimaged target region is determined, virtual ultrasonic imagedata may be displayed as a model assuming that the unimaged targetregion is imaged. The virtual ultrasonic image data may be generated bysynthesizing pixel values at positions close to the unimaged targetregion in the already existing data area, or may be an MPR imagegenerated from the volume data when the CT image volume data and thelike already exist.

2. First Modification

The target in the subject imaged by the ultrasonic probe 30 may be anabnormal target. For example, the target in the subject to be imaged maybe a target including a tumor. In this case, the deriving function 172derives the target shape and the imaged target region in the samesubject from ultrasonic image data of cross-sections to be imaged andtheir position data on the basis of the past medical image data (forexample, ultrasonic image data) of the same subject. Then, thedetermining function 173 determines an unimaged target region based onthe target shape derived by the deriving function 172 and the imagedtarget region. The determining function 173 can extract the position ofthe tumor based on the past medical image data, or can determine theposition of the data area based on the position data of the ultrasonicimage data of the cross-sections to be imaged.

As a result, when the tumor part is not imaged, the output controlfunction 174 outputs three-dimensional information of the unimagedtarget region including the tumor determined by the determining function173 or information for imaging the unimaged target region including thetumor to the outside of the processing circuitry 17.

On the other hand, when the tumor target has been imaged, thedetermining function 173 determines the degree of data-missing in thetumor of the imaged target region. When it is determined that the degreeof data-missing in the tumor is greater than or equal to the thresholdvalue, the output control function 174 outputs information for imagingthe tumor in the imaged target region to the outside of the processingcircuitry 17. The display example of the information for imaging thetumor is the same as the above mentioned “display examples (1) to (4) ofthe information regarding the unimaged target region”. As a result, itis possible to encourage the operator to reimage the tumor at a fineinterval or from a suitable direction. If the degree of data-missing inthe tumor is less than the threshold value and it is determined that thedata related to the tumor is sufficient, the output control function 174allows to finish of ultrasonic imaging.

3. Second Modification

A case where the transmission control function 73 of the output controlfunction 174 outputs the three-dimensional information of the unimagedtarget region determined by the determining function 173 and/or theinformation for imaging the unimaged target region to the externalapparatus of the ultrasonic diagnostic apparatus 1 will be described.

FIG. 13 is a schematic diagram showing a configuration of a medicalimage system including the ultrasonic diagnostic apparatus as an exampleof a medical image diagnostic apparatus according to the secondmodification.

FIG. 13 shows a medical image system S including the ultrasonicdiagnostic apparatus 1 as the medical image diagnostic apparatus. Themedical image system S includes the ultrasonic diagnostic apparatus 1described above and a medical image display apparatus 80 as a medicalimage processing apparatus. The medical image display apparatus 80 is aworkstation that performs various types of image processing on medicalimage data, a portable information processing terminal such as a tabletterminal, or the like, and is connected to the ultrasonic diagnosticapparatus 1 so as to be communicable via the network N.

The medical image display apparatus 80 includes a network interface 86,processing circuitry 87, a memory 88, an input interface 89, and adisplay 90. The network interface 86, the processing circuit 87, thememory 88, the input interface 89, and the display 90 have the sameconfigurations as those of the network interface 16, the processingcircuitry 17, the main memory 18, the input interface 19, and thedisplay 20 shown in FIG. 1 , respectively, which description will beomitted.

The processing circuitry 87 reads and executes a computer program storedin the memory 88 or directly incorporated in the processing circuitry87, thereby realizes a display control function 71A. Hereinafter, thecase where the function 71A functions as software will be described asan example, but all or part of the function 71A may be provided in themedical image display apparatus 80 as a function of the circuit such asthe ASIC.

The display control function 71A includes a function of receiving theultrasonic image data acquired by the acquiring function 171 of theultrasonic diagnostic apparatus 1 and displaying such data on thedisplay 90, and a function of receiving the information regarding theunimaged target region and displays it on the display 90 when theunimaged target region is determined by the determining function 173 ofthe ultrasonic diagnostic apparatus 1. The display control function 71Amay also generate 3D hologram image data (stereoscopically visible imagedata), display it on a 3D hologram display as the display 90, andproject it on the body surface of the subject.

With the configuration shown in FIG. 13 , the medical image displayapparatus 80, which is the external apparatus of the ultrasonicdiagnostic apparatus 1, is possible to display three-dimensionalinformation of the unimaged target region and/or the information forimaging the unimaged target region.

According to at least one embodiment described above, it is possible todetermine the unimaged target region in medical image data. In addition,it is possible to reduce the insufficient imaging, thereby reducedifferences between operators and improve efficiency.

Further, in the above-described embodiment, the case where the organ inthe subject is the main target that the imaged organ region and theunimaged organ region are determined has been described. However, theorgan is merely an example of the target, and the same determination maybe performed on the target other than the organ in the subject. Forexample, the target may be a surgical device such as a stent inserted inthe subject instead of the organ, or may be a lesion such as a tumor orlymphoma, or a layer of muscle. The shape of the target to be imaged maybe collectively referred to as “target shape”. The imaged region may bereferred to as “imaged target region”. The unimaged region may bereferred to as “unimaged target region”.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel methods and systems describedherein may be embodied in a variety of other forms; furthermore, variousomissions, substitutions, and changes in the form of the methods andsystems described herein may be made without departing from the spiritof the inventions. The accompanying claims and their equivalents areintended to cover such forms or modifications as would fall within thescope and spirit of the inventions.

What is claimed is:
 1. A medical image diagnostic apparatus comprising:processing circuitry configured to sequentially acquire cross sectionalmedical images of a target of a subject acquired by performing animaging of the target by scanning with a scanner, and position data ofthe acquired cross sectional medical images, automatically determine,based on the acquired cross sectional medical images and the acquiredposition data, an unimaged region being a part of the target, theunimaged region being included in a data-missing area, which has a sizeequal to or larger than a threshold, and which appears between adjacentcross sectional medical images of the acquired cross sectional medicalimages, and automatically move the scanner such that the unimaged regionwithin the target is scanned when the unimaged region is determined. 2.The medical image diagnostic apparatus according to claim 1, wherein theprocessing circuitry is configured to further display informationregarding the unimaged region when the unimaged region is determined. 3.The medical image diagnostic apparatus according to claim 2, wherein theprocessing circuitry is configured to display the information regardingthe unimaged region after the imaging is completed.
 4. The medical imagediagnostic apparatus according to claim 2, wherein the processingcircuitry is configured to display the information regarding theunimaged region during imaging.
 5. The medical image diagnosticapparatus according to claim 2, wherein the processing circuitry isconfigured to display the unimaged region three-dimensionally or displayinformation for imaging the unimaged region, thereby displaying theinformation regarding the unimaged region.
 6. The medical imagediagnostic apparatus according to claim 2, wherein the processingcircuitry is configured to project the unimaged region and a positionfor imaging the unimaged region on the target, thereby displaying theinformation regarding the unimaged region.
 7. The medical imagediagnostic apparatus according to claim 2, wherein the processingcircuitry is configured to display the unimaged region and a positionfor imaging the region on a three-dimensional model as the informationregarding the unimaged region.
 8. The medical image diagnostic apparatusaccording to claim 1, wherein the processing circuitry is configured toderive a target shape of the subject and an imaged region of the targetincluded in the acquired cross sectional medical images from theacquired cross sectional medical images and position data thereof, anddetermine the unimaged region based on the derived target shape andimaged region.
 9. The medical image diagnostic apparatus according toclaim 1, wherein the processing circuitry is further configured to movea scanner when the unimaged region is determined.
 10. The medical imagediagnostic apparatus according to claim 1, wherein the processingcircuitry is further configured to store, in a storage medium,information for imaging with a three-dimensional information regardingthe unimaged region, and/or transmit the information for imaging withthe three-dimensional information regarding the unimaged region to anoutside of the medical image diagnostic apparatus.
 11. The medical imagediagnostic apparatus according to claim 1, wherein the processingcircuitry is configured to sequentially acquire the cross sectionalmedical images of an organ which is the target, acquired by performingan imaging of the organ, and automatically determine, based on theacquired cross sectional medical images and the acquired position data,an unimaged region being a part of the organ, which is not included inthe acquired cross sectional medical images.
 12. The medical imagediagnostic apparatus according to claim 1, wherein the processingcircuitry is configured to three-dimensionally arrange the acquiredcross sectional medical images based on the position data thereof,derive the target and an imaged region of the target included in thearranged cross sectional medical images based on the arranged crosssectional medical images and the position data thereof, andautomatically determine the unimaged region based on the derived targetand imaged region.
 13. The medical image diagnostic apparatus accordingto claim 1, wherein the data-missing area further appears on: (a) a deepside of a short-depth cross sectional medical image of the acquiredcross sectional medical images in a depth direction; (b) a side wherethere is no cross sectional medical image adjacent to an edge crosssectional medical image of the acquired cross sectional medical images;or (c) a side of any of the acquired cross sectional medical images in adirection orthogonal to the depth direction and a moving direction ofthe scanner.
 14. A ultrasonic diagnostic apparatus comprising:processing circuitry configured to sequentially acquire cross sectionalultrasonic images of a target of a subject acquired by performing animaging of the target by scanning with an ultrasonic probe, and positiondata of the acquired cross sectional ultrasonic images, automaticallydetermine, based on the acquired cross sectional ultrasonic images andthe acquired position data, an unimaged region being a part of thetarget, the unimaged region being included in a data-missing area, whichhas a size equal to or larger than a threshold, and which appearsbetween adjacent cross sectional medical images of the acquired crosssectional medical images, and automatically move the ultrasonic probesuch that the unimaged region within the target is scanned when theunimaged region is determined.
 15. The ultrasonic diagnostic apparatusaccording to claim 14, wherein the processing circuitry is configured toderive a target shape of the subject and an imaged region of the targetincluded in the acquired cross sectional ultrasonic images from theacquired cross sectional medical images and position data thereof, anddetermine the unimaged region based on the derived target shape andimaged region.
 16. A medical image system in which a medical imagediagnostic apparatus and a medical image processing apparatus arecommunicably connected via a network, comprising: processing circuitryconfigured to sequentially acquire cross sectional medical images of atarget of a subject acquired by performing an imaging of the target byscanning with a scanner, and position data of the acquired crosssectional medical images, automatically determine, based on the acquiredcross sectional medical images and the acquired position data, anunimaged region being a part of the target, the unimaged region beingincluded in a data-missing area, which has a size equal to or largerthan a threshold, and which appears between adjacent cross sectionalmedical images of the acquired cross sectional medical images, andautomatically move the scanner such that the unimaged region within thetarget is scanned when the unimaged region is determined.
 17. Themedical image system apparatus according to claim 16, wherein theprocessing circuitry is further configured to derive a target shape ofthe subject and an imaged region of the target included in the acquiredcross sectional medical images from the acquired cross sectional medicalimages and position data thereof, and determine the unimaged regionbased on the derived target shape and imaged region.
 18. An imagingcontrol method comprising: sequentially acquiring cross sectionalmedical images of a target of a subject acquired by performing animaging of the target by scanning with a scanner, and position data ofthe acquired cross sectional medical images; automatically determining,based on the acquired cross sectional medical images and the acquiredposition data, an unimaged region being a part of the target, theunimaged region being included in a data-missing area, which has a sizeequal to or larger than a threshold, and which appears between adjacentcross sectional medical images of the acquired cross sectional medicalimages; and automatically moving the scanner such that the unimagedregion within the target is scanned when the unimaged region isdetermined.
 19. The imaging control method according to claim 18,further comprising: deriving a target shape of the subject and an imagedregion of the target included in the acquired cross sectional medicalimages from the acquired cross sectional medical images and positiondata thereof, and determining the unimaged region based on the derivedtarget shape and imaged region.